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COMPUTER IMPLEMENTED SYSTEM MANAGEMENT TOOL AND 

METHOD 

Field of the Invention 

The present invention relates generally to the field of software tools for 
organizations and more particularly to a system management tool. 

Background of the Invention 

There are a number of softv^are tools for analyzing organizations and businesses. 
The most widely used and popular is the spreadsheet. Spreadsheets are used for 
accounting, budgeting, and analyzing numbers including some engineering projects. 
However, the spreadsheet is a general purpose product that does not integrate all the 
functions required by many organizational problems. Other products are designed for 
more specific purposes, such as financial accounting, cost accounting and project 
accounting problems. While these packages can be very helpfial, they all are concerned 
with the monetary accounting surrounding a problem. Obviously, monetary accounting of 
organizational problems is very important however, other parameters need to be tracked 
and monitored to get a complete picture of how an organization system is working. These 
parameters can include availability of subsystems, failure rates of subsystem, etc. Some 
of the software packages provide some ability to track other parameters; however, 
monetary parameters always seem to control the structure. 

Thus, there exists a need for a system management tool that is not driven by 
accounting concems but by management concerns. 
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Brief Description of the Drawings 

FIG. 1 is a block diagram of a system management tool in accordance with one 
embodiment of the invention; 
5 FIG. 2 is a block diagram of a system management tool in accordance with one 

embodiment of the invention; 

FIG. 3 is a screen shot of a home page of a system management tool in accordance 
with one embodiment of the invention; 

FIG. 4 is a screen shot of a database query in a system management tool in 
10 accordance with one embodiment of the invention; 

FIG. 5 is a screen shot of a database report in a system management tool in 
accordance with one embodiment of the invention; 

FIG. 6 is a screen shot of a graph of a metric in a system management tool in 
accordance with one embodiment of the invention; 
1 5 FIG. 7 is a screen shot of a graph of a metric in a system management tool in 

accordance with one embodiment of the invention; 

FIG. 8 is a screen shot of a graph of a metric in a system management tool in 
accordance with one embodiment of the invention; and 

FIG. 9 is a flow chart of the steps used in a method of operating a system 
20 management tool in accordance with one embodiment of the invention. 

FIG. 1 0 is a flow chart of the steps used in a method of operating a system 
management tool in accordance with one embodiment of the invention. 
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DetaUed Description of the Drawing 

A system management tool includes an input system. An electronic library is 
connected to the input system and includes a process diagram for modeling a system. An 
5 evaluation system is connected to the electronic library. A corrective action system is 

connected to the evaluation system. An assessment and analysis system is connected to 
the corrective action system. The assessment and analysis system measures and 
documents a corrective action. The tool is designed to monitor a system, which is defined 
in the process diagram. The tool not only includes an evaluation system but a corrective 

10 action system. An assessment and analysis system is used to determine how well the 

corrective action is being implemented. 

FIG. 1 is a block diagram of a system management tool 10 in accordance with one 
embodiment of the invention. An input system 12 is connected to an electronic library 14 
for entering therein data relating to the various systems and subsvstems (e.g., sensors) 

15 being monitored. The electronic library 14 stores all the data that is entered via the input 

system 12 . The input system 12 can be a terminal in which the data relating to the various 
systems and subsystems is manually entered or the data can be automatically received 
fi*om a subsystem in the system being monitored. The electronic library 14 also includes a 
process diagram 1 6 for the process modeling the systems and subsystem being monitored . 

20 The process diagram 1_6 describes all the inputs into and outputs from the system 

management tool 1 0 regarding the systems and subsvstems being monitored . An 
evaluation system 18 is connected to the electronic library 14. The evaluation system 18 
compares the sys te m's performance of the systems and subsystems being monitored firom 
data stored in the electronic library 14 against a number of different metrics (plurality of 

25 metrics) stored in electronic library 14 . Evaluation means to find the value or amount of. 

to judge or determine the worth or quality of what is evaluated, and so the evaluation 
system 18 is a system to find the value or amount of to judge or determine the worth or 
quality of what is being evaluated. A corrective action system 20 is connected to the 
evaluation system 18. The corrective action system 20 includes plans to correct concerns 

30 discovered by the comparisons against metrics performed by the evaluation system 1 8. 

The corrective action system 20 includes metrics to determine if the corrective action 
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worked. Corrective means tending or meant to correct or improve, a remediaU and so 
corrective action relates to the doing of something remedial or tending to or meant to 
correct or improve what is being considered, and so the corrective action system 20 is a 
system for the doing of something remedial or tending to or meant to correct or improve 
what is being considered. An assessment and analysis system 22 is connected to the 
corrective action system 20. The assessment and analysis system 22 measures any 
corrective actions provided by the corrective action system 20 . Assessment means the act 
of assessing or of estimating or determining the significance, importance or value of in 
this instance, performance relative to a goal or of a sensor. Analysis means a separating 
or breaking up of any whole into its parts, especially with an examination of these parts to 
find out their nature, proportion, fijnction, interrelationship, and the like. Assessment and 
analysis system 22 is a system for assessing or estimating or determining the significance, 
importance or value of, in this instance, performance relative to a goal or of a sensor, such 
as by separating or breaking up of any whole into its parts, especially with an examination 
of these parts to find out their nature, proportion, function, interrelationship, and the like. 
Financial information is treated as just another input or output of the system being 
monitored by the system management tool 10. As a result, the system management tool 
10 is not limited to or by an accounting system mentality. 

FIG. 2 is a block diagram of a system management tool 30 in accordance with one 
embodiment of the invention. A user of the system management tool 30 may access the 
system management tool 30 using a network (e.g., hitemet— World Wide Web). A user 
having a computer 32 with a web browser system connects in system management tool 30 
through a network 34 to a web page server 36. An access control system 38 limits access 
to the web page server 36. In one embodiment, the access control system 38 requires a 
correct password be entered before the user is allowed access to web page server 36 and 
elements connected thereto for displaying data therefi-om on the monitor of the web 
accessing computer 32 . In other embodiments, fiirther security measures are 
implemented. The web page server 36 is connected to an electronic library 40. The 
electronic library 40, in this embodiment, includes both data fi-om the system or systems 
being monitored and documents that are required by personnel working on the system or 
systems being monitored . An evaluation system 42 is connected to the web page server 
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36. A corrective action system 44 is connected to the web page server 36. An assessment 
and analysis system 46 is connected to the corrective action system 44 via web server 36. 
The assessment and analysis system 46 measures any corrective actions provided by the 
corrective action system 44. By having the system management tool 30 designed for a 
5 network, it allows the system management tool 30 to be shared by multiple users in many 

different locations , e.g. wherever a computer with web browser 32 is available . 

A system management tool 10, 30 comprises an input system 12, an electronic 
library 14, 40 connected to the input system 12 and including a process diagram 16 for 
modeling a system, an evaluation system 18, 42 connected to the electronic library 14, 40, 

10 a corrective action system 20, 44 connected to the evaluation system 1 8, 42, and an 

assessment and analysis system 22, 46 connected to the corrective action system 20, 44, 
the assessment and analysis system 22, 46 measuring a corrective action. The tool 10, 30 
may include a report writing wizard with cut and paste functions from the electronic 
library 12, 40, the evaluation system 18, 42, the corrective action system 20, 44 and the 

15 assessment and analysis system 22, 46. The tool may include a plurality of interactive 

links 82-94 connected to the evaluation system 18, 42, and the interactive links 82-94 may 
include a central data repository 84 (part of electronic library 14, 40) storing all the data 
for the system. A plurality of hyperlinks 82-94 may provide access to the electronic 
library 14, 40, the evaluation system 18, 42 and the corrective action system 20, 44. The 

20 evaluation system 18, 42 may include a plurality of metrics 98, 102 and a plurality of 

associated thresholds 1 00, and may include a graphing system for displaying each of the 
plurality of metrics 98, 102 and each of the plurality of associated thresholds 100. The 
corrective action system 20, 44 may include a forecasting tool 66, 92. 

A system management tool 10, 30 comprises a web browser system 32, a network 

25 34 connected to the web browser system 32, an access control system 38 connected to the 

network 34, a web page server 36 connected to the access system 38, an electronic library 
14, 40 accessible fi'om the web page server 36, an evaluation system 18, 42 accessible 
from the web page server 36: and a corrective action system 20, 44 accessible fi-om the 
web page server 36. The tool 10, 30 may include an assessment and analysis system 22, 

30 46 accessible fi'om the web page server 36, the assessment and analysis system 22, 46 

measuring a corrective action. The electronic library 14, 40 may include a process 
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diagram 16, 80 defining an input and an output. The tool 10, 30 may include a report 
writing system 112 accessible by the web browser system 32. The tool 10, 30 may 
include interactive links 82-94 accessible fi-om the web page server 36 containing a data 
repository 84. 

5 FIG. 3 is a screen shot of a home page 60 of a system management tool 10, 30 in 

accordance with one embodiment of the invention. The home page 60 is designed to have 
a graphical look that emphasizes the fianctions of the tool system management tool 10, 30. 
e.g., using graphical symbols (icons') arranged in a logical manner and connected by 
arrows suggestive of a block diagram or system diagram of the system management tool 

10 10, 30 . The graphical symbols are hyperlinks. Library 62 and Interactive Links 82 

provide icons for hyperlinks to various databases and sources of information described 
below, and Decision Support Indicators 96, Features 104 and TPE Assessment 120 
regions provide icons for hyperlinks to tools described below useful to a user of system 
10, 30 for navigation fi-om home page 60. 

15 An electronic library [[62]] 62, 84 including library 62 provides access to a 

number of useful documents and reports e.g., via links fi-om home page 60 . For instance, 
this embodiment, has hyperlinks presented as icons 62-80 to link to archived reports 64, 
forecasting model instructions 66, sensor technical baseline information 68, data 
repository instructions 70, metric definitions and collection plan 72, Technical 

20 Performance Evaluation (TPE) operating instruction 74, decision support system 

documentation 76, Qualification Test and Evaluation (QT&E) Procedures 78 and TPE 
Process Diagrams 80. This iconic presentation with hyperlinks allows the electronic 
library 62 to contain most of the reference material that may be required by users of the 
system or systems and to be presented together in a defined region or sub-page of home ' 

25 page 60 . Interactive links (plurality of interactive links) 82 are next presented together in 

a defined region or sub-page adjacent to the library 62 and includes the data repository 
(central data repository) 84. Other databases 86 are also accessible via links fi-om home 
page 60, e.g., a database identified by the acronym ESR, the meaning of which is not 
relevant to the system management tool 10, 30 or necessary to understand the operation 

30 thereof as described herein . In addition, other tools such as databases 88, 90 are 

accessible via links fi-om home page 60. e.g., tools idenfified by the acronyms PDMS, 
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MIDIS and RAMES, the meanings of which are not relevant to the function of system 
management tool 10. 30 and are not necessary to understanding the system management 
tool 10, 30 described herein . A forecasting model (forecasting tool) 92 is accessible and 
other links 94 may also be accessible via links fi-om home page 60 . A decision support 
5 indicators section (evaluation system) 96 is next to the interactive links 82. 

Decision Support Indicators 96 are presented together in a defined region or sub- 
page adjacent to the interactive links 82 region or sub-page. Under the decision support 
indicators 96 are hyperlinks represented by icons 98-102 representing forecasting 
indicators 98, threshold indicators 100 and unstable process indicators 102 , e.g.. also 
10 referred to as unstable metrics . Next to the decision support indicators 96 is a features 

section 104. 

Features 104 are presented together in a defined region or sub-page adjacent to the 
Decision Support Indicators 96 region or sub-page. The features section 104 includes 
hyperlinks represented by icons 106-1 18 representing a number of different options, such 

15 as email 1 06 icon to open a window for sending an e-mail message and search 1 08 icon to 

open a window from which to utilize searching capabilities. A courses of action section 
1 1 0 icon opens a window that is used to determine and document corrective actions , e.g., 
to determine previous corrective actions and to document new corrective actions . A 
report writing feature (report writing system) 112 icon accesses a report writing feature 

20 that simplifies the process of collecting data and information for reports 1 12 , since all the 

required data and information is part of the system 10, 30, e.g., accessed via hyperlinks of 
library 62 and of interactive links 82, and can be cut and pasted (e.g., electronically 
selected, copied and inserted) into a report 1 12 . In addition, the reports section 112 has a 
report writing wizard that contains the basic format and layout for many required reports. 

25 The features section 104 also contains icons for hyperlinks accessing a Decision Support 

System (DSS) options section 114, on-line help section 116 and metric relationships 
section 118. A TPE assessment section 120 is next to the features section 104. TPE 
Assessment features 120 are presented together in a defined region or sub-page adjacent 
to the Features 1 04 region or sub-page. The TPE assessment section 1 20 includes 

30 hyperlinks represented by icons 122-124 representing a goal assessment system 122 and a 

sensor assessment section 1 24. 
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FIG. 4 is a screen shot 140 of a database query in a system management tool in 
accordance with one embodiment of the invention. The screen shot 140 query is of the 
ESR Database link 86 which provides information, e.g., on monthly outages for each site. 
In this example, the 2V^ Space Wing ESR Report is the subiect of the query and 
5 information from the ESR Report of information provided by the 21^' Space Wing 

organization is presented on screen shot 140. A drop-down menu 142 is provided to 
select a time period for the query, e.g., "Nov 99.'' The screen shot 140 shows that the user 
has selected the November 1999 ESR report which has a 79% availability rating. The 
user may run the report by clicking on the Run ESR Report button 144 or may cancel the 

10 query of ESR Database 86 by clicking on the Cancel button 146. Screen shot 140 may be 

accessed by clicking on the ESR Database link 86, e.g., the ESR Database link 86 found 
on screen shot 60 of FIG. 3 or on screen shot 160 of FIG. 6. 

FIG. 5 is a screen shot of a database report 1 50 in a system management tool 10^ 
30 in accordance with one embodiment of the invention. The user receives the report 

1 5 shown in FIG. 5 titled STATUS REPORT MISSION relating to Operational Availability 

(Ao) for November 1999 in response to selecting a month and year from drop down menu 
142 and clicking on the Run ESR Report button 144 of screen shot 140 of FIG. 4 . This 
allows the user to understand why the system received a 79% availability rating for the 
month of November 1999 in this example . The report screen 1 50 shows on the right-hand 

20 side thereof the total time 1 52 and dat e that the system was down and the start and stop 

times and dates 1 54 the system was down , e.g., dates and times in November 1999 in this 
example . The remarks section 156 explains why the system was down and a reference 
1 58 to the number of the related ESR Report is also provided . A closer look at the 
r emarks Remarks section 156 shows the system including elements Spacetrack and FPS- 

25 00 was down f^^RED^') mainly to test new software versions identified as "ESC DT&E^' . 

A manager might use this information to justify equipment to test the software other than 
by using the operational system , i.e. a corrective action . 

The Status Report of screen shot 150 includes on the left-hand side thereof 
information 159 defining the systems and subsystems to which the information on the 

30 right-hand side thereof pertains. Among this information 1 59 that may be present in this 

report format 1 50 is information that may be displayed under various headings, for 
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example, the site fSITE). equipment (EQUIPS channel fCHN) and work unit code 
rWUC), as well as mission impact codes fMIC, MICB. etc.), relating to the systems and 
subsystems, such as Spacetrack and FPS-00, listed under Remarks. The foregoing 
meanings of the example abbreviations and acronyms are known to users of an 
5 embodiment of system management tool 10, 30, but are examples not necessary to 

understanding the operation thereof. Where a system is noted as being "RED" under 
Remarks 156, the red or down condition is indicated by the letter "R" in various ones of 
the information 159 headings at the left-hand side the Status Report of screen shot 150. 
Screen shot 150 includes a scroll bar along the right hand edge thereof for easily scrolling 

10 through the information provided thereby, when the number of selections exceeds what 

can be displayed in screen shot 150. 

FIG. 6 is a screen shot 160 of an example report fi:'om Data Repository 84 of 
Interactive Links 82 which includes a graph 162 of a metric in a system management tool 
10, 30 in accordance with one embodiment of the invention. The screen shot 160 shows a 

1 5 graph (graph system) of the operational availability (Ao) of a sensor site 1 62. A threshold 

164 of 95% is shown on the graph and on the Threshold link 1 69 located beneath the 
graph . It is easy to see that the displayed example sensor site is rarely making its goal of 
95% operational availability (Ao) . 

Screen shot 160 provides a menu bar 61 and other hyperlinks, indicated by 

20 underlined symbols, words and phrases, for conveniently navigating to other parts of 

system management tool 10, 30, as may be desired by a user thereof without having to 
return to home page 60. Interactive Links 82 is highlighted on menu bar 61 to indicate 
that screen shot 160 is among the selections available under the Interactive Links 82 
heading on home page 60. For example, just below the menu bar 61 are hyperlinks to 

25 Metric Definitions and Collection Plan 72. to View Sensor Site Excel Workbook 85 of 

data repository 84, to ESR Database 86 and to Threshold Indicators History 100. The 
View Sensor Site Excel Workbook link 85 is a link to view an Excel spread sheet 
workbook of data relating to the selected Sensor Site. Threshold indicators history link 
100 is to a history of the threshold indicators relating to the data displayed 162 on screen 

30 shot 160. 

Along the left of screen shot 160 are provided menu lists 165. 166, and buttons for 
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selecting the data to be displayed in graph window 162. For example. Select a Sensor 
menu list 1 65 provides for the selection of a sensor and/or site for which data is to be 
displayed, e.g., fi^om among sensors and sites identified as OC3F. Socorro, GLOBUS IL 
PARCS, PAVE PAWS, Beale, Cape Cod, and Sensor Site, in this example. A scroll bar 
5 is provided at the right side of menu list 165 for scrolling to find a desired sensor on that 

list, when the number of selections exceeds what can be displayed in window 165. As 
indicated by their being indented in the list of menu 1 65, Beale and Cape Cod are sites 
under the PAVE PAWS sensor and OC3F and Socorro are sites under a sensor whose 
name is above the lines of menu list 165 displayed in the window thereof in this example. 

10 In this example. Sensor Site is selected as indicated by the highlighting of that selection in 

menu list 165 and by the heading information displayed near the top of graph window 
162. Sensor Site and the other entries in menu list 165 comprise examples of systems and 
subsystems monitored and/or managed utilizing system management tool 10, 30 

A Select a Metric menu list 166 is provided for selecting among various metrics, 

15 e.g.. Number of Hardware Upgrades, Number of Messages Generated, Number of Version 

Releases, Objects, Operational Availability (Ao). Projected Cost Savings, ODRs and 
Scheduled Downtime Rate (AmV A scroll bar is provided at the right side of menu list 
166 for scrolling to find a desired metric on that list, when the number of selections 
exceeds what can be displayed in window 166. In addition, selection buttons indicated bv 

20 circles and underlined links are provided for selecting the information to be displayed in 

menu list 166. A user may select a metric Oabel/instruction indicated by lack of 
underhne) or may select to Customize Favorites (underlined link) fi-om among the 
available metrics. A user may select to view all metrics (selected as indicated by filled in 
circle) or to view a list of favorite metrics (unfilled circle), which list of favorites may be 

25 customized (underlined link). In this example. Operational Availability Ao is selected 

from menu list 166 as indicated by the highlighting of that selection in menu list 166 and 
by the heading information displayed near the top of graph window 162 Further, an 
asterisk associated with the entry Operational Availability Ao indicates via a displayed 
note below menu list 166 that the selected metric is a metric that is an ESR Database 86 

30 Metric. The metrics listed in window 166 comprise examples of ones of the plurality of 

metrics available for monitoring and/or managing systems and subsystems ufilizing 
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system management tool 10, 30. 

In graphical display area 162 is a heading indicating the selected sensor fi-om 
menu list 165 and the selected metric fi'om menu list 166. In the graph 162, the vertical 
scale is percentage of Operational Availability Ao which is selected from menu list 166 
5 and the horizontal scale is the time period by month and year. The vertical scale is 

selected for suitable display of the range of values of the metric presented, e.g., from 50% 
to 100%. The selected Operational Availability Ao metric and the Threshold level 164 
are displayed differently in graph 162 so as to be easily discerned by a user and a legend 
thereof is provided beneath the graph 162, e.g., indicating that Operational Availability 

10 Ao is displayed by a solid line and the Threshold by a dashed line. Desirably, a prominent 

arrow is provided indicating the direction of displayed data of graph 162 that is deemed 
"good" and specified by the word "GOOD" therein, e.g., in this example, a higher value 
of the Ao metric is considered to be good. The Threshold indication 1 69 near the bottom 
of graph area 162 is underlined indicating it is a link that can be used for displaying 

15 information relating to the threshold level that is displayed on the graph 162 for the 

selected metric and sensor, as well as indicating the threshold level that is displayed. 

Thus, system management tool 10, 30 may be used to collect and organize data 
and metrics from a plurality of diverse sensors and sites, and to present such data in a 
plurality of formats and combinations that are useful for management. 

20 FIG. 7 is a screen shot 1 70 of a graph 172 of a metric in a system management 

tool 10, 30 in accordance with one embodiment of the invention. This screen shot 170 
shows a graph 172 of an inherent availability (Ai) of a sensor site 1 72. The target or 
threshold 174 for this metric is about 99.5%. This sensor site is generally meeting this 
metric requirement. If this is the same sensor site as shown in FIG. 6, then it would be 

25 important to detennine what the difference (i.e. relationship) is between these two 

metrics , since graph 162 appears to show poorer performance than does graph 172 . The 
screen shot 170 lists those metrics related to inherent availability (Ai) and provides for 
comparison of related metrics. This screen shot 170 contributes to the electronic library 
14, 40 , evaluation system 18, 42 , assessment and analysis system 22, 46 and the 

30 corrective action system 20, 44 . The electronic library 14, 40 has a definition for all these 

metrics. 
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Screen shot 170 provides a menu bar 61 and other hyperlinks, indicated bv 
underlined symbols, words and phrases, for conveniently navigating to other parts of 
system management tool 10, 30. as may be desired bv a user thereof without having to 
return to home page 60. Features 104 is highlighted on menu bar 61 to indicate that 
5 screen shot 170 is among the selections available under the Features 104 heading on home 

page 60. and the heading Metric Relationships indicates that selection 1 1 8 thereunder has 
been selected. For example, just below the menu bar 61 is a hyperlink to Metric 
Definitions and Collection Plan 72. 

A Select a Relationship menu list 175 is provided for selecting among various 

10 related metrics, e.g.. Inherent Availability (Ai\ MTBCF. MTTRS - Restore System. 

Operational Availability (Ao). Scheduled Downtime Rate (Am). System Availability 
(As), and Unscheduled Downtime Rate TAr). Selection buttons indicated by circles and 
underlined links are provided for selecting the information to be displayed in menu list 
175. A user may select a metric relationship flabel/instruction indicated by lack of 

15 underline) from among Predefined Groups (selected as indicated by filled in circle) of 

related metrics or from among Custom Groups (unfilled circle) of related metrics, or may 
select Create Relationship (underlined link) to create a customized group of metrics from 
among the available metrics. In this example. Inherent Availability Ai is selected from 
the group of related metrics displayed in menu list 175 as indicated bv the highlighting of 

20 that selection in menu list 175 and as also indicated by the heading information displayed 

near the top of graph window 172. The metrics listed in window 175 comprise examples 
of ones of the plurality of metrics available for monitoring and/or managing systems and 
subsystems utilizing system management tool 10. 30. 

A user may select from drop down menu 1 76 a source of the related metric data 

25 selected (highlighted) from among the related metrics displayed in menu list 1 75. In this 

example, metrics for the 21st Space Wing have been selected in drop down menu 176 and 
a description of the selected metric for the selected organization is displayed in the 
window 1 77 below menu 1 76 indicating that the source of the selected related metric data 
is the 21st Space Wing ESR Database 86. A user may also Select a Sensor (e.g.. system) 

30 from drop down menu 178 in which Sensor Site is selected in this example, as in the 

example of FIG. 6. from the list of available selections therein, e.g.. the same list as 
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available in menu 165 of screen shot 160. 

In graphical display area 172 is a heading indicating the selected sensor, which is 
Sensor Site in this example, from menu list 178 and the selected related metric, which is 
Inherent Availability Ai in this example, fi-om menu list 175. hi the graph 172, the 
5 vertical scale is percentage of Inherent Availability Ai which is selected from menu list 

1 75 and the horizontal scale is the time period by month and year. The vertical scale is 
selected for suitable display of the range of values of the metric presented, e.g., from 95% 
to 100%. The selected Inherent Availability Ai metric and the Threshold level are 
displayed differently in graph 172 so as to be easily discemed by a user and a legend 

10 thereof is provided beneath the graph 172, e.g., indicating that Inherent Availability Ai is 

displayed by a solid line and the Threshold by a dashed line. Desirably, a prominent 
arrow is provided indicating the direction of displayed data of graph 172 that is deemed 
"good" and specified by the word "GOOD" therein, e.g., in this example, a higher value 
of the Inherent Availability Ai metric is considered to be good. The Threshold indication 

15 179 near the bottom of graph area 172 is underlined indicating it is a link that can be used 

for displaying information relating to the threshold level that is displayed on the graph 
1 72 for the selected metric and sensor. 

FIG. 8 is a screen shot 1 80 of a graph 182 of a metric in a system management 
tool 10, 30 in accordance with one embodiment of the invention. The screen shot 180 

20 shows a bar chart 182 of hardware fault dis t ribution d e nsi t y 182 distribution/density 

between February 1993 and March 2000. It is easy to see that subsystem 1-1 184 is the 
major source of faults. This screen shot 180 lists those metrics related to hardware fault 
dis t ribu t ion density distribution/density and provides for comparison of related metrics. 
This screen shot 180 contributes to the electronic library 14, 40 , evaluation system 18, 42 , 

25 assessment and analysis system 22. 46 and the corrective action system 20, 44 . 

Screen shot 180 provides a menu bar 61 and other hyperlinks, indicated by 
underlined symbols, words and phrases, for conveniently navigating to other parts of 
system management tool 10, 30, as may be desired by a user thereof without having to 
return to home page 60. Features 104 is highlighted on menu bar 61 to indicate that 

30 screen shot 180 is among the selections available under the Features 104 heading on home 

page 60, and the heading Metric Relationships indicates that selection 1 1 8 thereunder has 
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been selected. For example, just below the menu bar 61 is a hyperlink to Metric 
Definitions and Collection Plan 72. Screen shot 180 is the same screen shot as screen 
shot 1 70 described above except that a different group of related metrics and a different 
data source have been selected, and the data is presented in bar graph format 1 82. Menu 
5 list 175, pull down menus 176. 178 and description window 177 are the same as in screen 

shot 170 expect that the information therein corresponds to the different selections of 
screen shot 1 80. 

A Select a Relationship menu list 175 is provided for selecting among various 
related metrics, e.g.. Hardware Fault Distribution/Density. MTBF. MTBMA and MTTR- 

10 Repair. Selection buttons indicated by circles and underlined links are provided for 

selecting the information to be displayed in menu list 1 75. A user may select a metric 
relationship flabel/instruction indicated by lack of underline) fi"om among Predefined 
Groups (selected as indicated by filled in circle) of metrics or firom among Custom 
Groups (unfilled circle) of metrics, or may select Create Relationship ( underlined link) to 

15 create a customized group of metrics fi^om among the available metrics. In this example. 

Hardware Fault Distribution/Density is selected firom menu list 175 as indicated by the 
highlighting of that selection in menu list 175 and by the heading information displayed 
near the top of graph window 182. The metrics listed in window 175 comprise examples 
of ones of the plurality of metrics available for monitoring and/or managing systems and 

20 subsystems utilizing system management tool 10, 30. 

A user may select fi-pm drop down menu 1 76 a source of the related metric data 
selected in menu list 175, in this example the metrics fi-om the RAMES database 90 
having been selected in drop down menu 176 and a description thereof is displayed in the 
window 177 below menu 176 indicating that the source of the selected related metric data 

25 is the RAMES Database 90. Infonnation in window 177 may also contain information 

relating to availability and limitation of the metrics data, for this example that the data is 
based on data from another source (e.g., CAMS inputs) and that only limited data is 
available (e.g., data fi-om PAVE PAWS sensor sites and fi-om BMEWS sensor sites is 
limited). A user may also Select a Sensor (e.g., a system) fi-om drop down menu 178 in 

30 which Sensor Site is selected in this example, as in the examples of FIGS. 6 and 7, fi-om 

the list of available selections therein, e.g., the same list as available in menu 165 of 
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screen shot 1 60. 

hi graphical display area 182 is a heading indicating the selected sensor, which is 
Sensor Site in this example, from menu list 178 and the selected related metric, which is 
Hardware Fault Distribution/Density in this example, from menu list 1 75. In the graph 
5 182, the yertical scale is the number of failures or faults of Hardware Fault 

Distribution/Density as selected from menu list 175 and the horizontal scale is a listing of 
Sensor Site and of the various sub-systems 184 comprising the system Sensor Site. The 
time period by month and year is also provided in the heading of graphical display area 
1 82 because the date information is not the horizontal axis in this example. The vertical 

10 scale is selected for suitable display of the range of values of the metric presented, e.g., 

from 0 to 1500 failures. The selected Hardware Fault Distribution/Density metric 
produces data for each subsystem that is not related to the data for another subsystem and 
so are displayed in graph 182 as bars of varying height representing the number of 
failures, so as to be easily compared by a user. The bars of bar graph 1 82 are displayed 

15 from left to right in order of descending height, thereby to more graphically portray the 

relative data for the various subsystems. The bar representing the number of Sensor Site 
failures is necessarily higher because it represents the total of the other bars, i.e. because 
Sensor Site is the example system comprising the example subsystems listed along the 
horizontal axis . A legend 185 thereof is provided beneath the graph 182, e.g., a dark box 

20 185 indicating that Hardware Fault Distribution/Density is displayed by a bar of like 

character. Desirably, a prominent arrow 183 is provided indicating the direction of 
displayed data of graph 1 82 that is deemed "good" and specified by the word "GOOD" 
therein, e.g., in this example, a lower value of the Hardware Fault Distribution/Density 
metric is considered to be good. The Threshold indication 189 near the bottom of graph 

25 area 1 82 is underlined indicating it is a link that can be used for displaying infonnation 

relating to the threshold level that is displayed on the graph 182 for the selected metric 
and sensor. Sensor Site and the subsystems listed along the horizontal axis of graph 182 
comprise examples of systems and subsystems being monitored and/or managed utilizing 
system management tool 10, 30 

30 In the screen shots of each of FIGS. 4-8, a menu bar of the web browser of 

computer 32 is displayed on the display or monitor thereof, e.g., along the upper edge of 
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screen shot 140, 150, 160, 170. 180, and provides buttons/links for navigating among the 
various features and functions of the web browser of computer 32, e.g., buttons for 
forward, back, delete, home page and the like which are common to web browsers and are 
not used by system management tool 10, 30 which provides its own buttons, menus and 
5 links as described herein. 

hi the screen shots of each of FIGS. 6-8, a menu bar 61, e.g., along the upper edge 
of screen shot 160, 170, 180, provides buttons corresponding to home page 60 and the 
links 62, 82, 96, 104, 120 to sub-pages of home page 60. Clicking on a button 62, 82, 96, 
104. 120 provides a respective drop down menu or a sub-page for selecting among the 

10 various elements 64-80. 84-94. 98-102. 106-1 18 and 122-124 under elements 62, 82, 96, 

104. 120, respectively, of home page 60 of system management tool 10. 30. Simply 
clicking on the button corresponding to the desired element, e.g.. a button for accessing 
home page 60. a button for Library 62. a button for Interactive Links 82. a button for 
Decision Support Indicators 96.a button for Features 104 and a button for TPE 

15 Assessment 1 20, produces a drop down menu from which any selection thereunder may 

be accessed directly by clicking on the name thereof. A question mark button is 
provided on menu bar 61 for accessing the on-line Help function 116. 

The button 62. 82. 96. 104, 120 of menu bar 61 corresponding to the element of 
system 10. 30 in which the currently displayed function resides is outlined or highlighted 

20 to indicate that relationship. For example, when Data Repository 84 under Interactive 

Links 82 is displayed as in screen shot 160. the menu bar 61 button for Interactive Links 
82 is outlined or highlighted, as indicated by the double-lined menu bar 61 button 82 in 
FIG. 6. and when Metric Relationships 1 18 under Features 104 is displayed as in screen 
shots 170 and 180. the menu bar 61 button for Features 104 is outlined or highlighted, as 

25 indicated by the double-lined menu bar 61 button 104 in FIGS. 7 and 8. Home page 60 

need not have a menu bar 61 because icons 62, 82, 96, 104, 120 providing links to all the 
elements of system management tool 10, 30 are displayed along the upper region thereof, 
and on-line help 1 16 is also a selection option. 

Certain sensors that can be managed utilizing system management tool 10. 30 and 

30 certain sources of data utilized thereby are designated herein bv acronyms, such as PDMS, 

RAMES, PAVE PAWS, BMEWS and the like, and the meanings of these acronyms are 
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not relevant to the fiinction of the system management tool 10, 30 or necessary to 
understand the operation thereof as described herein. 

FIG. 9 is a flow chart of the steps used in a method of operating a system 
management tool 10, 30 in accordance with one embodiment of the invention. The 
5 process starts, step 200, by trarisrhitting a connection request over a network to a projec t 

34 to a system management server 36, which may include a web server 36 and an access 
control system 38 as shown in FIG. 2, at step 202. A fi-ont page fhome page 60) is 
received at step 204. The user then selects fi-om a pluraHty of options that include an 
electronic library 14, 40 , a data repository 84 or an evaluation system 18, 42, at step 206. 

10 When the data repository 84 is selected by clicking on a link at step 208, the user selects 

either to view data or enter data which e nds and the process ends at step 210. In one 
embodiment, the electronic library 14, 40 is selected and the user then selects from a 
process diagram, an instruction manual or a dictionary. When the evaluation system 18^ 
42 is selected the user can select a metric to analyze. A graph of the metric may be 

1 5 displayed. In one embodiment, the graph includes a threshold. When the evaluation 

system 18, 42 is selected, the user can choose between a set of metrics that exceed their 
threshold (plurality of associated thresholds). The user can also choose to view a set of 
metrics that are unstable. An unstable metric is a metric which may have process 
problems. 

20 A method 200 of operating a system management tool 10, 30 comprises the steps 

of transmitting 202 a connection request over a network 34 to a system management 
server 36, receiving 204 a fi-ont page 60, selecting 206 fi-om a plurality of options 62-124 
including an electronic library 14, 40, a data repository 84 and an evaluation system 18, 
42, and, when the data repository 84 is selected 208a by clicking on a link 82-94, 

25 selecting 208 either 208b to view data or 208c to enter data. The method 200 may include 

the step of selecting 224 a process diagram 80, selecting 226 an instruction manual 66, 74, 
76, 78, or selecting 228 a dictionary 76. 80, when the electronic library 14, 40 is selected 
206a. The method 200 may include the steps of selecting 212a metric to analyze when 
the evaluation system 18, 42 is selected 206b, and displaying 84. 92, 98, 100, 102, 214 a 

30 graph of the metric 162, 172, 182, and may also include the step 216 of displaying 84, 92, 

98, 100, 102 a threshold 164, 174. The method 200 may include the step of selecting 218 
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a set of metrics that exceed their threshold 100. 164. 174 when the evaluation system 18, 
42 is selected 206b. The method 200 may include the step of selecting 220 a set of 
metrics that are unstable 102 when the evaluation system 18. 42 is selected 206b. and/or 
selecting 222 a set of forecasting indicators 98 when the evaluation system 18. 42 is 
5 selected 206b. An unstable metric 102 is a metric which may have process problems 

(unstable process indicator). 

FIG. 10 is a flow chart of the steps used in a method 300 of operating a system 
management tool 10. 30 in accordance with one embodiment of the invention. Method 
300 is similar to method 200 and also includes steps described herein, e.g.. described in 

10 relation to various ones of FIGS. 2-8 and illustrated therein. Method 300 starts with the 

step 302 of transmitting a connection request over a network 34. e.g.. from a web browser 
of a computer 32 to a management server, e.g.. web server 36. via the Internet. World 
Wide Web, or other network 34. Step 303 comprises controlling access to the web page 
server 36 by an access control system 38 associated therewith that limits access to server 

15 36 by requiring a user identifier such as a password. If access is granted in step 303, step 

304 is the sending and receiving of a home page 60 from the web server 36 to the web 
browser 32 for display to the user on the monitor of web browser system 32. Home page 
60 presents a plurality of links to various fiinctions and options 62-124 that may be 
accessed by clicking on the icon or link name thereof. 

20 Step 306 comprises selecting from a plurality of options 62-124 of functions. 

databases and tools by clicking on the link thereto wherein the links are arranged under 
categories including library 62, interactive links 82. decision support links 96. features 
104 and TPE assessment 120. and wherein the links and icons are arranged to resemble a 
system diagram of system management tool 10. 30. e.g.. as shown in FIG. 3. 

25 When library 62 is selected 3 1 0. steps 311-319 comprise selecting one of archived 

reports 64. forecasting model instructions 66. sensor technical baseline information 68. 
data repository instructions 70. metric definitions and collection plan 72 TPE operating 
instructions 74. decision support system documentation 76. OT&E procedures 78 and 
TPE process diagrams 80. respectively. TPE process diagrams 80 may include process 

30 diagrams. Forecasting model instructions 66. data repository instructions 70. TPE 

operating instructions 74. decision support system documentation 76. and OT&E 
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procedures 78 may include an instruction manual. Metric definitions and collection plan 
72, decision support system documentation 76 and TPE process diagrams 80 may include 
a dictionary. Selections 360-369 to display 162, 172, 182 data or to fiirther select data to 
be displayed 162, 172, 182 may be made. 
5 When interactive links 82 is selected 320, steps 321-326 comprise selecting one of 

data repository 84, forecasting model database 92, PDMS tool 88 which includes an 
external database, ESR database 86, other interactive links 94 and RAMES tool 90 which 
includes an external database, respectively. When data repository 84 is selected 321, step 
321a comprises selecting to view data, e.g., as by screen shots 140-180, and step 321b 

10 comprises selecting to enter data, e.g., by input system 12 and/or web browser system 32. 

Then, selections 360-369 to display 162, 172, 182 data or to further select data to be 
displayed 162, 172, 182 maybe made. The data therein is useful for evaluation 18, 42. 

When decision support indicators 96 (e.g., evaluation system 18, 42) is selected 
330, steps 331-333 comprise selecting one of forecasting indicators 98, threshold 

15 indicators 100 and unstable process indicators (unstable metrics) 102, respectively. Then, 

selections 360-369 to display 162, 172, 182 data or to further select data to be displayed 
162, 172, 182 maybe made. Evaluation means to find the value or amount of, to Judge or 
determine the worth or quality of what is evaluated, and so the evaluation system 1 8, 42 is 
a system to find the value or amount of, to judge or determine the worth or quality of, 

20 what is being evaluated. Decision support indicators 96-102 are indicators, a device that 

measures or records and visibly indicates to support a decision, e.g., by a user. 
Forecasting indicators 98 are indicators that forecast, i.e. that estimate or calculate in 
advance, that predict or seek to predict, and threshold indicators 100 are indicators that 
relate to a threshold, i.e. the beginning point of something. Likewise, forecasting model 

25 92 and forecasting model instructions 66 therefor are models that estimate or calculate in 

advance, that predict or seek to predict. Decision support indicators 96-102 thus focus on 
indicating problems and potential problems for user consideration through various 
displayed information, e.g.. screen shots 140-180. 

When features 104 is selected 340, steps 341-347 comprise selecting one of 

30 courses of action feature 1 10, E-mail feature 106, on-line help feature 1 16, DSS options 

feature 1 14, metric relationships feature 118, preparing reports using reports wizard 1 12, 
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and searching system management tool 10. 30 feature 108. Then, selections 360-369 to 
display 1 62, 1 72, 182 data or to further select data to be displayed 1 62, 1 72, 1 82 may be 
made. 

Decision support indicators 96-102 and certain of features 104, e.g., search 108. 
5 courses of action 110, reports 1 12, and metric relationships 118, comprise corrective 

action system 20, 44. Corrective means tending or meant to correct or improve, remedial, 
and so corrective action relates to the doing of something remedial or tending to or meant 
to correct or improve what is being considered, and so the corrective action system 20 is a 
system for the doing of something remedial or tending to or meant to correct or improve 

10 what is being considered. 

When TPE assessment 120 (e.g., assessment and analysis system 22, 46) is 
selected 350, steps 351 and 352 comprise selecting one of goal assessment 122 and sensor 
assessment 124, respectively. Assessment means the act of assessing or of estimating or 
determining the significance, importance or value of, in this instance, performance 

15 relative to a goal or of a sensor. Analysis means a separating or breaking up of any whole 

into its parts, especially with an examination of these parts to find out their nature, 
proportion, function, interrelationship, and the like. Assessment and analysis system 22, 
46 is a system for assessing or estimating or determining the significance, importance or 
value of, in this instance, performance relative to a goal or of a sensor or a set of sensors, 

20 such as by separating or breaking up of any whole into its parts, especially with an 

examination of these parts to find out their nature, proportion, fianction, interrelationship, 
and the Hke. 

Step 360 comprises displaying a screen 140, 150, 160, 170, 180, in response to 
having selected 306, 310-352 a link 62-124. Displaying 360 a screen comprises providing 

25 Hnks and/or menus for steps 362-368 which comprise selecting any one or more of a time 

period 142, a month and year 142, a report 144, a sensor 165, 178, a site 165, 178, a 
metric 166, 175, a set of metrics 166, 175, a set of related metrics 166, 175, a threshold 
169, 179. 189. and a source of data 158, 176. When a time period 142, a month and year 
142, a report 144, a sensor 165, 178, a site 165, 178, a metric 166, 175, a set of metrics 

30 166, 175, a set of related metrics 166, 175, and/or a source of data 158, 176 have been 

selected 362-368, step 369 comprises displaying a graphical representation 162, 172. 182 
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of data relating thereto, wherein the graphical representation 162, 172, 182 may be in the 
form of a graphical plot 162, 172, or of a bar graph 182. Displaying 369 a graphical 
representation 162, 172, 182 may also include displaying a threshold level 164, 169, 174 
with the graphical plot 162, 172 and/or displaying a symbol 163, 173, 183 indicating a 
5 good condition, such as an arrow 163, 173, 183 having the word "good" therein. 

Bracket and line 370 indicates that the screens displayed in response to the 
selections 310-369 described above in relation to FIG. 10 include links, e.g., through 
menu bar 61 and/or by specific links on a particular screen (e.g., links to metric 
definitions and collection plan 72 on screens 160, 170, 180) to ones of the links selectable 

10 306 from home page 60 and firom the selections 310-369 thereunder. Thus, system 

management tool 10, 30 may be easily navigated fi"om a presently displayed screen to 
another screen without having to return to home page 60 each time. 

Thus, there has been described a system management tool that is not a slave to 
accounting systems but does address management concerns. The tool is designed to be 

1 5 used over a network, which allows widely dispersed users to access the tool. This makes 

the tool useful to both workers, mid-level managers and high-level managers and avoids 
repeating information throughout an organization. 

The methods described herein can be implemented as computer-readable 
instructions stored on a computer-readable storage medium that when executed by a 

20 computer will perform the methods described herein. A tool implemented on a computer 

is commonly referred to as a software tool or as an application, e.g., such as system 
management tool 10, 30, and tools 88, 90. 

While the invention has been described in conjunction with specific embodiments 
thereof, it is evident that many alterations, modifications, and variations will be apparent 

25 to those skilled in the art in light of the foregoing description. Accordingly, it is intended 

to embrace all such alterations, modifications, and variations in the appended claims. 



OAO-0001 -22- 

Substitute Specification 

(Marked Up Version - March 1 7, 2005) 



Patent Application 
No. 09/768,993 



Abstract 



A system management tool (10) includes an input system (12). An electronic 
libi aiy (14) is connec t ed t o the inpu t sys t em (12) and includes a process diagi ' am (16) for 
modeling a sys te m. An e valuation sys t em (18) is connected t o the electronic library (14). 
A coi T ective ac t ion syst e m (20) is comicc te d t o t he e valua t ion sys t em (18). An 
assessmen t and analysis system (22) is comiect e d t o the corrective action sys te m (20). 
Th e assessmen t and analysis system (22) measures and documents a correc t ive ac t ion and 
method is provided on a computer controlled by a machine readable instructions and 
comprises a library and plural external databases containing information relating to plural 
systems and/or subsystems. A home page provides links to the library, interactive links to 
the plural external databases: links to indicators for supporting a decision and to elements 
thereof including forecasting elements and unstable metrics, and links to technical 
performance evaluations and elements thereof including assessment elements. Screen 
shots displayed in response to selecting links on the home page provide selectable lists 
including at least a list of metrics and a list including ones of the systems and/or 
subsystems, and a graphical display for a selected metric and a selected system or 
subsystem of data for the graphical display is provided from the library, or the plurality of 
external databases, or both . 



